The present invention relates generally to the field of microscopy. More specifically, the present invention relates to a method and apparatus for three-dimensional deconvolution of optical microscope images.
It is known in light microscopy to use fluorescent dyes to study the spatial distribution of specific cellular elements. The use of fluorescent dyes allows examination of cellular elements that would not otherwise be discernible with conventional light microscopy. With the use of some very specialized dyes, specific cellular elements can be tagged for imaging, e.g., Hoechst 33258 is a dye for labeling DNA, see, Agard, D. A., Hiraoka, Y., Shaw, P., Sedat, J. W., Methods in Cell Biology--Volume 30, Chapter 13 "Fluorescence Microscopy in Three Dimensions," San Diego, Calif., Academic Press, pp. 353-377, 1989. A class of reversible cationic redistribution dyes, tetramethylrhodamine ethyl (TMRE) and methyl esters (TMRM) are also used. These dyes are used for the study of mitochondria since the dyes produce intensities proportional to membrane potentials, see, Loew, L. M., Tuft, R. A., Carrington, W., Fay, F. S., "Imaging in Five Dimensions: Time-Dependent Membrane Potentials in Individual Mitochondria," Biophysical Journal, Volume 65, pp. 2396-2407, December, 1993.
Three-dimensional imaging of a fluorescently dyed substrate is performed by collecting a series of x-y images along the an optical z-axis. The data is recorded by charge-coupled-device (CCD) arrays. The CCD array is a two-dimensional arrangement of CCD elements that collects an image or an image slice in the x-y plane. The optical focus of the system is moved to an adjacent plane where the next image slice in the x-y plane is acquired. This is continued until the desired number of planes (image slices) have been acquired for the three-dimensional data set.
Two types of microscope systems are typically used for image acquisition: a three-dimensional wide-field microscope and a scanning confocal microscope. Both of these microscope systems are considered to be diffraction-limited. Diffraction theory describes the effects of light passing through a finite aperture to an image plane. The resulting image displays the effects of the wave nature of light. The end result is that light emanates from a point as a spherical wave front and scatters through a finite aperture. Various treatments of diffraction theory have been presented, e.g., see Goodman, J. W., Introduction to Fourier Optics, New York, McGraw-Hill, pp. 101-136, 1988. The most commonly accepted treatments of diffraction theory are Fresnel approximations for near field (i.e., Fresnel diffraction) and Fraunhofer approximations for far field (i.e., Fraunhofer diffraction). In either case, the resulting image due to a diffraction-limited system can be described by the product of the Fourier transforms of a source image and aperture distribution. In other words, the resulting image is a result of a convolution of the source object's light with an aperture of the imaging system. Further, the system transfer-function is obtained directly by taking the Fourier transform of the aperture. It will be appreciated that the blurring effects due to convolution exist in two-dimensions only, i.e., the x-y planes. The point-spread-function (PSF) is the expression used to describe the convolutional blurring in two-dimensions. The PSF physically results from imaging a point source. The Fourier transform of the PSF is the system transfer-function, which is obtained by convolving the system transfer-function with a Dirac-delta function. A point source is the physical equivalent of a Dirac-delta function, and, in the frequency domain, the Dirac-delta function is a unity operator across the spectrum. Therefore, the Fourier transform of the PSF should be the Fourier transform of the aperture. However, the PSF contains noise and blurring due to other effects such as aberrations.
Achieving high resolution imaging requires the use of a high numerical aperture lens (on the order of N.A.=1.2 to 1.4 or greater). This is one of the basic problems with three-dimensional wide-field microscopy. A high numerical aperture lens, although needed to obtain high spatial resolution with adequate sensitivity, causes the undesirable effect of corrupting the data with out-of-focus light. The very large cone angle associated with a high N.A. lens results in a very limited depth-of-focus (e.g., possibly 0.4 .mu.m or less) , see, Agard, D. A., Hiraoka, Y., Shaw, P., Sedat, J. W., Methods in Cell Biology--Volume 30, Chapter 13 "Fluorescence Microscopy in Three Dimensions," San Diego, Calif., Academic Press, pp. 353-377, 1989. Consequently, such images are plagued with out-of-focus light from planes above and below the plane being imaged, resulting in reduced resolution and sensitivity of the three-dimensional wide-field data sets.
Of the aforementioned six types of aberrations (i.e., the five Seidel aberrations and the chromaticaberration), spherical aberrations are the most significant. Spherical aberrations have little effect in two-dimensional imaging but are very prevalent in three-dimensional imaging. This is due to the fact that objective lenses are designed to image at a specific plane, usually the immediate underside of the coverslip. Therefore, the other planes that are imaged show significant blurriness from spherical aberrations.
Another major contribution to blurriness is the point-spread-function (PSF). This is the actual subject of deconvolution, it is the only contributor to blurriness that is a result of a convolution. As discussed hereinbefore, PSF occurs as a separate process in each of the two-dimensional planes. The combination of these three different effects plus noise contributes to the overall blurriness in a three-dimensional data set. The filter, although called a deconvolution filter, is actually designed to filter out the effects of out-of-focus light, aberrations, PSF and noise all at the same time. The term deconvolution is somewhat of a misnomer since only the PSF blurriness requires deconvolving.
The imaging process is greatly improved by using a scanning confocal microscope, wherein a laser is illuminated through a pinhole focused on a point in an object. The point is, in turn, focused through a pinhole to a detector. The term confocal describes the fact that the illumination and detection are commonly focused to the same point in an object. The object is scanned in three-dimensions to generate the three-dimensional data set. The same lens may be used to achieve confocal conditions.
Methods for improving the resolution of the scanning confocal microscope over what can be achieved with conventional microscopy are known. By way of example, assuming a circular aperture, the intensity distribution of the main lobe is reduced in width by a factor of 1.389 compared to conventional microscopy, see, Agard, D. A., Hiraoka, Y., Shaw, P., Sedat, J. W., Methods in Cell Biology--Volume 30, Chapter 13 "Fluorescence Microscopy in Three Dimensions," San Diego, Calif., Academic Press, pp. 353-377, 1989. In three-dimensions, this translates to (1.389).sup.3 or an improvement of 2.64 in the voxel intensity distribution. This improvement in resolution directly relates to the amount of information that can be recorded in the data set. Also, the sidelobe intensity of the scanning confocal microscope is reduced from approximately 1% in conventional microscopy to 0.01% in confocal microscopy, again see, , Agard, D. A., Hiraoka, Y., Shaw, P., Sedat, J. W., Methods in Cell Biology--Volume 30, Chapter 13 "Fluorescence Microscopy in Three Dimensions," San Diego, Calif., Academic Press, pp. 353-377, 1989. This greatly reduces the convolution effects thus essentially eliminating scattering. As a result, the dynamic range of intensity is increased two orders of magnitude, from 100:1 in conventional microscopy to 10,000:1 in a confocal system.
In fluorescence microscopy the point resolution is broadened due to the increased wavelength of the fluorescent light compared to the shorter wavelength of the laser illumination light. Accordingly, resolution of the confocal microscope is somewhat reduced as a result of fluorescence, but this is also a problem with conventional fluorescence microscopy as well.
While, the illumination pinhole in a confocal microscope can be small enough so that the illumination is essentially a point source, the detection pinholes must typically be larger to collect enough light from weakly fluorescing objects. Consequently, the advantages gained in reducing convolution effects are somewhat diminished since the detected light is a result of the convolution with the pinhole aperture.
Image acquisition with the confocal microscope must be carried out on a point by point basis, either by scanning a stationary specimen or by moving the specimen with respect to the confocal pinholes. In any case, image acquisition is presently a relatively slow process.
Confocal microscopes suffer from photobleaching and phototoxicity. This occurs due to the high intensity laser light being focused on a point in the specimen. The fluorescent dyes contribute to produce toxic oxygen radicals within the cell. As is well known, photobleaching can be minimized with the use of antioxidants, however, this is chemically intrusive, thereby limiting the study of cell processes.
In light of the foregoing, when attempting to image dynamic cell processes, a three-dimensional wide-field microscope employing digital signal processing techniques is often preferred. Nevertheless, it is otherwise preferred to acquire high resolution images without the need for employing digital signal processing techniques using confocal microscopes.
Microscopy systems are used to acquire images with sub-micron resolution. In general, optical microscopy is limited in resolution by the wavelength of light (.lambda.) and the numerical aperture of the lens (N.A.=.eta. sin .alpha.), whereby the minimum resolvable distance between distinct points can be expressed as: ##EQU1## where .eta. is the index of refraction and .alpha. is a lens cone 1/2 angle, see Gray, P., Slayter, E. M., The Encyclopedia of Microscopy and Microtechnique, New York, Van Nostrand Reinhold, pp. 382-389, 1973, which is incorporated herein by reference. This limitation is due to lens aberrations causing out of focus blur and the effects of the convolution of an image with an aperture of a system. The convolution effects are limited to two-dimensions only. Therefore, the blurriness associated with a three-dimensional data set is due to convolution effects in two-dimensions (i.e., x-y image planes) and aberrations in three-dimensions. By way of example, for a circular aperture, the convolution of a point is the Airy disc pattern (i.e., a function of J.sub.1 which is a Bessel function of the first kind, first order). As is readily apparent, a further analysis leads to the Rayleigh resolution criteria which defines the minimum resolvable distance between two points as the separation of the peak intensities where the peak intensity of the first point corresponds with the first minimum of the second point, assuming a circular aperture. See, for example: Cherry, R. J., New Techniques of Optical Microscopy and Microspectroscopy, Boca Raton, Fla., CRC Press (Macmillan Press), pp. 31-43, 1991: Goodman, J. W., Introduction to Fourier Optics, New York, McGraw-Hill, pp. 101-136, 1988; and Gray, P., Slayter, E. M., The Encyclopedia of Microscopy and Microtechnique, New York, Van Nostrand Reinhold, pp. 382-389, 1973. The Rayleigh resolution criteria can be expressed as: ##EQU2## or, expressed in terms of the highest spatial frequency, f.sub.p, as: EQU Rayleigh d.sub.min =1.22/f.sub.p (3)
where f.sub.p is defined as: EQU f.sub.p =2.eta. sin .alpha./.lambda. (4).
It will be appreciated that the Rayleigh resolution criteria only addresses the convolution effects and is a two-dimensional phenomenon. However, aberrations affect the image quality in three-dimensions. The five types of aberrations in monochromatic systems (see, Smith, W. J., Modern Optical Engineering, New York, McGraw-Hill, pp. 49-58, 1966), commonly referred to as the Seidel aberrations are: spherical, astigmatism, curvature of field, distortion, and coma. If white light is used in the imaging process then the additional effects of chromatic aberration also degrade image quality. Lens are generally designed to greatly minimized aberrations. The convolution effects can be minimized as well by using a confocal microscope. Unfortunately, a confocal microscope would cause photobleaching and photodamage to cells thereby limiting its widespread application, as is well known. Consequently, three-dimensional wide-field microscopy of the prior art relies on digital signal processing (DSP) techniques to deconvolve an image data and remove out-of-focus blurring.
Prior attempts to use deconvolution have been plagued by poor signal-to-noise (S/N) results. One of the suspected reasons for these poor results are less than ideal convergence to the optimum deconvolution transfer function and the inability to adequately suppress noise. Typically, a Wiener filter implemented in the frequency-domain is used, but Wiener filters are notorious for not adequately suppressing noise, see Agard, D. A., Hiraoka, Y., Shaw, P., Sedat, J. W., Methods in Cell Biology--Volume 30, Chapter 13 "Fluorescence Microscopy in Three Dimensions," San Diego, Calif., Academic Press, pp. 353-377, 1989 and Cherry, R. J., New Techniques of Optical Microscopy and Microspectroscopy, Boca Raton, Fla., CRC Press (Macmillan Press), pp. 31-43, 1991. One reason for this is that the noise characteristics must be known a priori and are assumed to be statistically stationary. Any non-stationarities present in the data have an adverse effect on the signal-to-noise ratio (S/N). The non-stationarities can be dealt with by using more sophisticated algorithms, e.g., Kalman filters do not assume stationarity and could be used to track changing statistics. Wiener filters are a special case of the Kalman filter where stationarity is assumed. However, the difference in computational intensity often precludes the use of Kalman filters. The three-dimensional microscope data sets exhibit stationary noise characteristics indicating the validity of Wiener filtering techniques for this problem.